Skip to content

fix(external/sdk): propagate mapToStruct errors instead of silently dropping outputs#565

Merged
intel352 merged 4 commits into
mainfrom
copilot/fix-silent-drop-outputs
May 6, 2026
Merged

fix(external/sdk): propagate mapToStruct errors instead of silently dropping outputs#565
intel352 merged 4 commits into
mainfrom
copilot/fix-silent-drop-outputs

Conversation

Copilot AI commented May 6, 2026

Copy link
Copy Markdown
Contributor

plugin/external/sdk/grpc_server.go::mapToStruct suppressed structpb.NewStruct errors via s, _ = structpb.NewStruct(m), returning nil/empty instead of surfacing the failure. Any Outputs or service result containing a structpb-incompatible type (chan, func, unserializable struct) would silently vanish at the gRPC boundary — downstream consumers saw an empty map and treated it as "no changes."

plugin/external/convert.go was already fixed with the correct signature; this PR applies the same fix to the SDK's parallel copy.

Changes

  • sdk/grpc_server.gomapToStruct: changed signature to (*structpb.Struct, error); removed _ error suppression; wraps with fmt.Errorf("structpb.NewStruct: %w", err)
  • sdk/grpc_server.go — call sites (ExecuteStep, InvokeService ×2): handle error and surface it in Response.Error (consistent with existing //nolint:nilerr error-in-response-body pattern)
  • sdk/grpc_server_test.go: added mustMapToStruct test helper for known-good fixture data; updated existing call sites; added TestMapToStruct_SDK_PropagatesError and TestInvokeService_PropagatesOutputEncodingError
  • docs/PLUGIN_ARCHITECTURE.md: updated documented signature
// Before
func mapToStruct(m map[string]any) *structpb.Struct {
    s, _ := structpb.NewStruct(m) // error silently dropped
    return s
}

// After
func mapToStruct(m map[string]any) (*structpb.Struct, error) {
    s, err := structpb.NewStruct(m)
    if err != nil {
        return nil, fmt.Errorf("structpb.NewStruct: %w", err)
    }
    return s, nil
}

Copilot AI requested review from Copilot and removed request for Copilot May 6, 2026 04:43
@github-actions

github-actions Bot commented May 6, 2026

Copy link
Copy Markdown

⏱ Benchmark Results

No significant performance regressions detected.

benchstat comparison (baseline → PR)
## benchstat: baseline → PR
baseline-bench.txt:260: parsing iteration count: invalid syntax
baseline-bench.txt:283902: parsing iteration count: invalid syntax
baseline-bench.txt:592831: parsing iteration count: invalid syntax
baseline-bench.txt:865643: parsing iteration count: invalid syntax
baseline-bench.txt:1147614: parsing iteration count: invalid syntax
baseline-bench.txt:1459879: parsing iteration count: invalid syntax
benchmark-results.txt:260: parsing iteration count: invalid syntax
benchmark-results.txt:340470: parsing iteration count: invalid syntax
benchmark-results.txt:663832: parsing iteration count: invalid syntax
benchmark-results.txt:1012346: parsing iteration count: invalid syntax
benchmark-results.txt:1348047: parsing iteration count: invalid syntax
benchmark-results.txt:1625674: parsing iteration count: invalid syntax
goos: linux
goarch: amd64
pkg: github.com/GoCodeAlone/workflow/dynamic
cpu: AMD EPYC 7763 64-Core Processor                
                            │ benchmark-results.txt │
                            │        sec/op         │
InterpreterCreation-4                 3.406m ± 162%
ComponentLoad-4                       3.533m ±   0%
ComponentExecute-4                    1.917µ ±   1%
PoolContention/workers-1-4            1.071µ ±   1%
PoolContention/workers-2-4            1.073µ ±   2%
PoolContention/workers-4-4            1.076µ ±   1%
PoolContention/workers-8-4            1.075µ ±   1%
PoolContention/workers-16-4           1.087µ ±   3%
ComponentLifecycle-4                  3.528m ±   1%
SourceValidation-4                    2.233µ ±   0%
RegistryConcurrent-4                  785.6n ±   3%
LoaderLoadFromString-4                3.540m ±   1%
geomean                               17.32µ

                            │ benchmark-results.txt │
                            │         B/op          │
InterpreterCreation-4                  2.027Mi ± 0%
ComponentLoad-4                        2.180Mi ± 0%
ComponentExecute-4                     1.203Ki ± 0%
PoolContention/workers-1-4             1.203Ki ± 0%
PoolContention/workers-2-4             1.203Ki ± 0%
PoolContention/workers-4-4             1.203Ki ± 0%
PoolContention/workers-8-4             1.203Ki ± 0%
PoolContention/workers-16-4            1.203Ki ± 0%
ComponentLifecycle-4                   2.183Mi ± 0%
SourceValidation-4                     1.984Ki ± 0%
RegistryConcurrent-4                   1.133Ki ± 0%
LoaderLoadFromString-4                 2.182Mi ± 0%
geomean                                15.25Ki

                            │ benchmark-results.txt │
                            │       allocs/op       │
InterpreterCreation-4                   15.68k ± 0%
ComponentLoad-4                         18.02k ± 0%
ComponentExecute-4                       25.00 ± 0%
PoolContention/workers-1-4               25.00 ± 0%
PoolContention/workers-2-4               25.00 ± 0%
PoolContention/workers-4-4               25.00 ± 0%
PoolContention/workers-8-4               25.00 ± 0%
PoolContention/workers-16-4              25.00 ± 0%
ComponentLifecycle-4                    18.07k ± 0%
SourceValidation-4                       32.00 ± 0%
RegistryConcurrent-4                     2.000 ± 0%
LoaderLoadFromString-4                  18.06k ± 0%
geomean                                  183.3

cpu: AMD EPYC 9V74 80-Core Processor                
                            │ baseline-bench.txt │
                            │       sec/op       │
InterpreterCreation-4              3.140m ± 203%
ComponentLoad-4                    3.463m ±   3%
ComponentExecute-4                 1.819µ ±   1%
PoolContention/workers-1-4         1.019µ ±   1%
PoolContention/workers-2-4         1.024µ ±   2%
PoolContention/workers-4-4         1.019µ ±   1%
PoolContention/workers-8-4         1.020µ ±   1%
PoolContention/workers-16-4        1.022µ ±   1%
ComponentLifecycle-4               3.488m ±   1%
SourceValidation-4                 2.085µ ±   1%
RegistryConcurrent-4               773.8n ±   2%
LoaderLoadFromString-4             3.527m ±   0%
geomean                            16.59µ

                            │ baseline-bench.txt │
                            │        B/op        │
InterpreterCreation-4               2.027Mi ± 0%
ComponentLoad-4                     2.180Mi ± 0%
ComponentExecute-4                  1.203Ki ± 0%
PoolContention/workers-1-4          1.203Ki ± 0%
PoolContention/workers-2-4          1.203Ki ± 0%
PoolContention/workers-4-4          1.203Ki ± 0%
PoolContention/workers-8-4          1.203Ki ± 0%
PoolContention/workers-16-4         1.203Ki ± 0%
ComponentLifecycle-4                2.183Mi ± 0%
SourceValidation-4                  1.984Ki ± 0%
RegistryConcurrent-4                1.133Ki ± 0%
LoaderLoadFromString-4              2.182Mi ± 0%
geomean                             15.25Ki

                            │ baseline-bench.txt │
                            │     allocs/op      │
InterpreterCreation-4                15.68k ± 0%
ComponentLoad-4                      18.02k ± 0%
ComponentExecute-4                    25.00 ± 0%
PoolContention/workers-1-4            25.00 ± 0%
PoolContention/workers-2-4            25.00 ± 0%
PoolContention/workers-4-4            25.00 ± 0%
PoolContention/workers-8-4            25.00 ± 0%
PoolContention/workers-16-4           25.00 ± 0%
ComponentLifecycle-4                 18.07k ± 0%
SourceValidation-4                    32.00 ± 0%
RegistryConcurrent-4                  2.000 ± 0%
LoaderLoadFromString-4               18.06k ± 0%
geomean                               183.3

pkg: github.com/GoCodeAlone/workflow/middleware
cpu: AMD EPYC 7763 64-Core Processor                
                                  │ benchmark-results.txt │
                                  │        sec/op         │
CircuitBreakerDetection-4                     283.8n ± 8%
CircuitBreakerExecution_Success-4             21.56n ± 0%
CircuitBreakerExecution_Failure-4             65.78n ± 0%
geomean                                       73.83n

                                  │ benchmark-results.txt │
                                  │         B/op          │
CircuitBreakerDetection-4                    144.0 ± 0%
CircuitBreakerExecution_Success-4            0.000 ± 0%
CircuitBreakerExecution_Failure-4            0.000 ± 0%
geomean                                                 ¹
¹ summaries must be >0 to compute geomean

                                  │ benchmark-results.txt │
                                  │       allocs/op       │
CircuitBreakerDetection-4                    1.000 ± 0%
CircuitBreakerExecution_Success-4            0.000 ± 0%
CircuitBreakerExecution_Failure-4            0.000 ± 0%
geomean                                                 ¹
¹ summaries must be >0 to compute geomean

cpu: AMD EPYC 9V74 80-Core Processor                
                                  │ baseline-bench.txt │
                                  │       sec/op       │
CircuitBreakerDetection-4                 298.1n ± 13%
CircuitBreakerExecution_Success-4         22.67n ±  0%
CircuitBreakerExecution_Failure-4         70.98n ±  0%
geomean                                   78.27n

                                  │ baseline-bench.txt │
                                  │        B/op        │
CircuitBreakerDetection-4                 144.0 ± 0%
CircuitBreakerExecution_Success-4         0.000 ± 0%
CircuitBreakerExecution_Failure-4         0.000 ± 0%
geomean                                              ¹
¹ summaries must be >0 to compute geomean

                                  │ baseline-bench.txt │
                                  │     allocs/op      │
CircuitBreakerDetection-4                 1.000 ± 0%
CircuitBreakerExecution_Success-4         0.000 ± 0%
CircuitBreakerExecution_Failure-4         0.000 ± 0%
geomean                                              ¹
¹ summaries must be >0 to compute geomean

pkg: github.com/GoCodeAlone/workflow/module
cpu: AMD EPYC 7763 64-Core Processor                
                                 │ benchmark-results.txt │
                                 │        sec/op         │
JQTransform_Simple-4                        864.8n ± 28%
JQTransform_ObjectConstruction-4            1.442µ ±  0%
JQTransform_ArraySelect-4                   3.261µ ±  0%
JQTransform_Complex-4                       38.29µ ±  0%
JQTransform_Throughput-4                    1.760µ ±  1%
SSEPublishDelivery-4                        74.44n ±  1%
geomean                                     1.653µ

                                 │ benchmark-results.txt │
                                 │         B/op          │
JQTransform_Simple-4                      1.273Ki ± 0%
JQTransform_ObjectConstruction-4          1.773Ki ± 0%
JQTransform_ArraySelect-4                 2.625Ki ± 0%
JQTransform_Complex-4                     16.22Ki ± 0%
JQTransform_Throughput-4                  1.984Ki ± 0%
SSEPublishDelivery-4                        0.000 ± 0%
geomean                                                ¹
¹ summaries must be >0 to compute geomean

                                 │ benchmark-results.txt │
                                 │       allocs/op       │
JQTransform_Simple-4                        10.00 ± 0%
JQTransform_ObjectConstruction-4            15.00 ± 0%
JQTransform_ArraySelect-4                   30.00 ± 0%
JQTransform_Complex-4                       324.0 ± 0%
JQTransform_Throughput-4                    17.00 ± 0%
SSEPublishDelivery-4                        0.000 ± 0%
geomean                                                ¹
¹ summaries must be >0 to compute geomean

cpu: AMD EPYC 9V74 80-Core Processor                
                                 │ baseline-bench.txt │
                                 │       sec/op       │
JQTransform_Simple-4                     825.4n ± 25%
JQTransform_ObjectConstruction-4         1.423µ ±  1%
JQTransform_ArraySelect-4                3.347µ ±  0%
JQTransform_Complex-4                    40.55µ ±  1%
JQTransform_Throughput-4                 1.735µ ±  0%
SSEPublishDelivery-4                     65.22n ±  2%
geomean                                  1.619µ

                                 │ baseline-bench.txt │
                                 │        B/op        │
JQTransform_Simple-4                   1.273Ki ± 0%
JQTransform_ObjectConstruction-4       1.773Ki ± 0%
JQTransform_ArraySelect-4              2.625Ki ± 0%
JQTransform_Complex-4                  16.22Ki ± 0%
JQTransform_Throughput-4               1.984Ki ± 0%
SSEPublishDelivery-4                     0.000 ± 0%
geomean                                             ¹
¹ summaries must be >0 to compute geomean

                                 │ baseline-bench.txt │
                                 │     allocs/op      │
JQTransform_Simple-4                     10.00 ± 0%
JQTransform_ObjectConstruction-4         15.00 ± 0%
JQTransform_ArraySelect-4                30.00 ± 0%
JQTransform_Complex-4                    324.0 ± 0%
JQTransform_Throughput-4                 17.00 ± 0%
SSEPublishDelivery-4                     0.000 ± 0%
geomean                                             ¹
¹ summaries must be >0 to compute geomean

pkg: github.com/GoCodeAlone/workflow/schema
cpu: AMD EPYC 7763 64-Core Processor                
                                    │ benchmark-results.txt │
                                    │        sec/op         │
SchemaValidation_Simple-4                      1.113µ ± 10%
SchemaValidation_AllFields-4                   1.705µ ± 18%
SchemaValidation_FormatValidation-4            1.623µ ±  3%
SchemaValidation_ManySchemas-4                 1.823µ ±  2%
geomean                                        1.539µ

                                    │ benchmark-results.txt │
                                    │         B/op          │
SchemaValidation_Simple-4                      0.000 ± 0%
SchemaValidation_AllFields-4                   0.000 ± 0%
SchemaValidation_FormatValidation-4            0.000 ± 0%
SchemaValidation_ManySchemas-4                 0.000 ± 0%
geomean                                                   ¹
¹ summaries must be >0 to compute geomean

                                    │ benchmark-results.txt │
                                    │       allocs/op       │
SchemaValidation_Simple-4                      0.000 ± 0%
SchemaValidation_AllFields-4                   0.000 ± 0%
SchemaValidation_FormatValidation-4            0.000 ± 0%
SchemaValidation_ManySchemas-4                 0.000 ± 0%
geomean                                                   ¹
¹ summaries must be >0 to compute geomean

cpu: AMD EPYC 9V74 80-Core Processor                
                                    │ baseline-bench.txt │
                                    │       sec/op       │
SchemaValidation_Simple-4                   1.123µ ±  2%
SchemaValidation_AllFields-4                1.704µ ± 13%
SchemaValidation_FormatValidation-4         1.635µ ±  4%
SchemaValidation_ManySchemas-4              1.627µ ±  1%
geomean                                     1.502µ

                                    │ baseline-bench.txt │
                                    │        B/op        │
SchemaValidation_Simple-4                   0.000 ± 0%
SchemaValidation_AllFields-4                0.000 ± 0%
SchemaValidation_FormatValidation-4         0.000 ± 0%
SchemaValidation_ManySchemas-4              0.000 ± 0%
geomean                                                ¹
¹ summaries must be >0 to compute geomean

                                    │ baseline-bench.txt │
                                    │     allocs/op      │
SchemaValidation_Simple-4                   0.000 ± 0%
SchemaValidation_AllFields-4                0.000 ± 0%
SchemaValidation_FormatValidation-4         0.000 ± 0%
SchemaValidation_ManySchemas-4              0.000 ± 0%
geomean                                                ¹
¹ summaries must be >0 to compute geomean

pkg: github.com/GoCodeAlone/workflow/store
cpu: AMD EPYC 7763 64-Core Processor                
                                   │ benchmark-results.txt │
                                   │        sec/op         │
EventStoreAppend_InMemory-4                   1.280µ ± 18%
EventStoreAppend_SQLite-4                     1.525m ±  6%
GetTimeline_InMemory/events-10-4              14.13µ ±  4%
GetTimeline_InMemory/events-50-4              78.70µ ±  2%
GetTimeline_InMemory/events-100-4             125.4µ ± 12%
GetTimeline_InMemory/events-500-4             641.7µ ±  1%
GetTimeline_InMemory/events-1000-4            1.322m ±  1%
GetTimeline_SQLite/events-10-4                106.8µ ±  0%
GetTimeline_SQLite/events-50-4                249.6µ ±  0%
GetTimeline_SQLite/events-100-4               431.2µ ±  1%
GetTimeline_SQLite/events-500-4               1.813m ±  0%
GetTimeline_SQLite/events-1000-4              3.537m ±  2%
geomean                                       225.2µ

                                   │ benchmark-results.txt │
                                   │         B/op          │
EventStoreAppend_InMemory-4                     800.5 ± 8%
EventStoreAppend_SQLite-4                     1.984Ki ± 2%
GetTimeline_InMemory/events-10-4              7.953Ki ± 0%
GetTimeline_InMemory/events-50-4              46.62Ki ± 0%
GetTimeline_InMemory/events-100-4             94.48Ki ± 0%
GetTimeline_InMemory/events-500-4             472.8Ki ± 0%
GetTimeline_InMemory/events-1000-4            944.3Ki ± 0%
GetTimeline_SQLite/events-10-4                16.74Ki ± 0%
GetTimeline_SQLite/events-50-4                87.14Ki ± 0%
GetTimeline_SQLite/events-100-4               175.4Ki ± 0%
GetTimeline_SQLite/events-500-4               846.1Ki ± 0%
GetTimeline_SQLite/events-1000-4              1.639Mi ± 0%
geomean                                       67.42Ki

                                   │ benchmark-results.txt │
                                   │       allocs/op       │
EventStoreAppend_InMemory-4                     7.000 ± 0%
EventStoreAppend_SQLite-4                       53.00 ± 0%
GetTimeline_InMemory/events-10-4                125.0 ± 0%
GetTimeline_InMemory/events-50-4                653.0 ± 0%
GetTimeline_InMemory/events-100-4              1.306k ± 0%
GetTimeline_InMemory/events-500-4              6.514k ± 0%
GetTimeline_InMemory/events-1000-4             13.02k ± 0%
GetTimeline_SQLite/events-10-4                  382.0 ± 0%
GetTimeline_SQLite/events-50-4                 1.852k ± 0%
GetTimeline_SQLite/events-100-4                3.681k ± 0%
GetTimeline_SQLite/events-500-4                18.54k ± 0%
GetTimeline_SQLite/events-1000-4               37.29k ± 0%
geomean                                        1.162k

cpu: AMD EPYC 9V74 80-Core Processor                
                                   │ baseline-bench.txt │
                                   │       sec/op       │
EventStoreAppend_InMemory-4                1.089µ ± 12%
EventStoreAppend_SQLite-4                  1.007m ±  5%
GetTimeline_InMemory/events-10-4           12.23µ ±  2%
GetTimeline_InMemory/events-50-4           64.31µ ± 18%
GetTimeline_InMemory/events-100-4          106.5µ ±  2%
GetTimeline_InMemory/events-500-4          543.9µ ±  1%
GetTimeline_InMemory/events-1000-4         1.109m ±  2%
GetTimeline_SQLite/events-10-4             83.48µ ±  1%
GetTimeline_SQLite/events-50-4             219.0µ ±  0%
GetTimeline_SQLite/events-100-4            383.6µ ±  1%
GetTimeline_SQLite/events-500-4            1.670m ±  0%
GetTimeline_SQLite/events-1000-4           3.259m ±  1%
geomean                                    189.3µ

                                   │ baseline-bench.txt │
                                   │        B/op        │
EventStoreAppend_InMemory-4                 759.0 ± 13%
EventStoreAppend_SQLite-4                 1.984Ki ±  1%
GetTimeline_InMemory/events-10-4          7.953Ki ±  0%
GetTimeline_InMemory/events-50-4          46.62Ki ±  0%
GetTimeline_InMemory/events-100-4         94.48Ki ±  0%
GetTimeline_InMemory/events-500-4         472.8Ki ±  0%
GetTimeline_InMemory/events-1000-4        944.3Ki ±  0%
GetTimeline_SQLite/events-10-4            16.74Ki ±  0%
GetTimeline_SQLite/events-50-4            87.14Ki ±  0%
GetTimeline_SQLite/events-100-4           175.4Ki ±  0%
GetTimeline_SQLite/events-500-4           846.1Ki ±  0%
GetTimeline_SQLite/events-1000-4          1.639Mi ±  0%
geomean                                   67.12Ki

                                   │ baseline-bench.txt │
                                   │     allocs/op      │
EventStoreAppend_InMemory-4                  7.000 ± 0%
EventStoreAppend_SQLite-4                    53.00 ± 0%
GetTimeline_InMemory/events-10-4             125.0 ± 0%
GetTimeline_InMemory/events-50-4             653.0 ± 0%
GetTimeline_InMemory/events-100-4           1.306k ± 0%
GetTimeline_InMemory/events-500-4           6.514k ± 0%
GetTimeline_InMemory/events-1000-4          13.02k ± 0%
GetTimeline_SQLite/events-10-4               382.0 ± 0%
GetTimeline_SQLite/events-50-4              1.852k ± 0%
GetTimeline_SQLite/events-100-4             3.681k ± 0%
GetTimeline_SQLite/events-500-4             18.54k ± 0%
GetTimeline_SQLite/events-1000-4            37.29k ± 0%
geomean                                     1.162k

Benchmarks run with go test -bench=. -benchmem -count=6.
Regressions ≥ 20% are flagged. Results compared via benchstat.

Copilot AI requested review from Copilot and removed request for Copilot May 6, 2026 05:54
Copilot AI changed the title [WIP] Fix silent drop of Outputs on structpb.NewStruct errors fix(external/sdk): propagate mapToStruct errors instead of silently dropping outputs May 6, 2026
Copilot AI requested a review from intel352 May 6, 2026 05:56
@intel352
intel352 marked this pull request as ready for review May 6, 2026 06:00
Copilot AI review requested due to automatic review settings May 6, 2026 06:00

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR updates the external plugin SDK gRPC server to stop silently dropping step/service outputs when structpb.NewStruct fails, and instead propagates encoding errors back to the caller via the response Error field. This aligns the SDK’s behavior with the already-fixed plugin/external/convert.go path and prevents “empty output means no changes” misinterpretations at the gRPC boundary.

Changes:

  • Update mapToStruct in the SDK gRPC server to return (*structpb.Struct, error) and propagate structpb.NewStruct failures.
  • Handle encoding errors in ExecuteStep and InvokeService call sites by returning response-level errors.
  • Add/adjust SDK tests to validate error propagation and update docs to reflect the new signature.

Reviewed changes

Copilot reviewed 3 out of 3 changed files in this pull request and generated 2 comments.

File Description
plugin/external/sdk/grpc_server.go Propagates structpb.NewStruct errors and surfaces them via response Error fields at gRPC boundaries.
plugin/external/sdk/grpc_server_test.go Adds tests/helpers to ensure encoding failures are not silently dropped and updates existing tests for new signature.
docs/PLUGIN_ARCHITECTURE.md Updates documented mapToStruct signature to include error return.

})
if err != nil {
t.Fatalf("CreateModule rpc error: %v", err)
}
Comment thread plugin/external/sdk/grpc_server_test.go Outdated
@intel352

intel352 commented May 6, 2026

Copy link
Copy Markdown
Contributor

@copilot apply changes based on the comments in this thread

Copilot AI requested review from Copilot and removed request for Copilot May 6, 2026 13:44

Copilot AI commented May 6, 2026

Copy link
Copy Markdown
Contributor Author

@copilot apply changes based on the comments in this thread

Done in e16166b:

  • Added createResp.Error != "" and createResp.HandleId == "" guards in TestInvokeService_PropagatesOutputEncodingError so a CreateModule failure surfaces as its own distinct fatal rather than leaking into the InvokeService assertion.
  • Fixed mustPackGRPCTestMessage formatting — t.Helper() is now on its own line, matching the rest of the file.

@intel352
intel352 merged commit 58688d6 into main May 6, 2026
18 checks passed
@intel352
intel352 deleted the copilot/fix-silent-drop-outputs branch May 6, 2026 13:56
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

plugin/external/convert.go:mapToStruct silently drops Outputs on structpb.NewStruct errors

3 participants